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EVOLVING  NOVEL  COEFFICIENT  SETS  FOR  OPTIMIZED 
RECONSTRUCTION  OF 

QUANTIZED  ONE-DIMENSIONAL  (1-D)  AND  TWO-DIMENSIONAL  (2-D) 

SIGNALS 


Abstract.  This  paper  describes  a  genetic  algorithm  that  evolves  optimized  sets  of 
coefficients  for  signal  reconstruction  under  lossy  conditions  due  to  quantization. 
Beginning  with  a  population  of  mutated  copies  of  the  set  of  coefficients  describing  a 
wavelet-based  inverse  transform,  our  genetic  algorithm  systemically  evolves  a  new  set  of 
coefficients  that  significantly  reduces  mean  squared  error  (relative  to  the  performance  of 
the  selected  wavelet)  for  various  classes  of  1-D  and  2-D  signals. 


1  INTRODUCTION 

Wavelets  [1]  are  commonly  used  to  solve  problems  drawn  from  a  wide  range  of 
application  areas.  Wavelet  transforms  associated  with  orthonormal,  compactly  supported 
wavelets  [2]  and  biorthogonal  wavelets  [3]  have  been  shown  to  achieve  signal 
compression  ratios  as  high  as  10:1,  50:1,  and  even  100:1  without  significant  information 
loss.  In  these  cases,  the  corresponding  inverse  wavelet  transforms  are  capable  of 
reconstructing  very  close  approximations  of  the  original  signal. 

For  many  practical  problems,  however,  it  becomes  necessary  to  represent  a  given  signal 
using  a  smaller  range  of  possible  values.  For  example,  telephone  signals  (for  which  speed 
of  transmission  is  most  important)  are  represented  by  as  few  as  8  bits,  while  music 
signals  (which  require  higher  signal  fidelity)  are  typically  represented  by  16-bit  values. 
Quantization  [4]  is  the  process  of  mapping  signals  to  a  smaller  number  of  bits.  Figure  1 
illustrates  the  process  of  reconstructing  quantized  1-D  signals.  A  2-D  transform  is 
accomplished  by  performing  a  1-D  transform  on  each  row  of  the  image,  followed  by  a 
1-D  transform  on  each  column. 


v(k) 


DWT 


0"  hKff)- 


DWT1 


v’(k) 


Figure  1. 1-D  Reconstruction  Discrete  Wavelet  Transform  (DWT)  Filter  with 

Quantization 

Errors  introduced  into  the  transformed  signal  via  quantization  may  have  an  unacceptably 
adverse  effect  upon  the  quality  of  the  signal  when  it  is  subsequently  reconstructed  via  the 
wavelet  inverse  transform.  A  growing  amount  of  empirical  evidence  (e.g.,  [5])  suggests 
that  nontraditional  inverse  transformations  may  do  a  better  job  of  compensating  for  the 
negative  effects  of  quantization,  resulting  in  higher  fidelity  signal  reconstruction.  In 
particular,  recent  studies  suggest  that  the  use  of  adaptive  ([6],  [7])  and/or  nonstandard  [8] 
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filters  may  significantly  reduce  errors  for  specific  classes  of  signals,  which  typically 
share  spatial  domain  characteristics. 


2  METHODOLOGY 

Genetic  algorithms  (GAs)  are  optimization  techniques  inspired  by  Darwinian  evolution. 
GAs  [9]  have  been  successfully  applied  to  an  ever-increasing  number  of  difficult  and 
interesting  optimization  problems.  The  goal  of  this  investigation  was  to  develop  a  GA 
capable  of  automatically  modifying  the  coefficient  sets  describing  wavelet  inverse 
transform  functions  [10]  to  evolve  novel  inverse  transforms  exhibiting  significantly 
improved  performance  for  a  given  class  of  signals  [1 1],  In  particular,  our  GA 
automatically  compensated  for  errors  introduced  into  the  original  signal  by  quantization. 
Performance  may  be  measured  in  many  ways  [12];  for  this  study,  performance  equaled 
the  mean  squared  error  (MSE)  in  the  reconstructed  signal. 

Figure  2  illustrates  our  GA-based  inverse  transform  optimization  process.  The  best-of-run 
inverse  transform  coefficients  produced  by  the  GA  are  used  to  replace  the  DWT"1  shown 
in  Figure  1.  To  improve  upon  wavelet-based  techniques,  our  GA  had  to  evolve  optimized 
inverse  transform  coefficients  that  significantly  reduced  the  aggregate  MSE  in  each 
reconstructed  signal  v’(k). 


Figure  2.  Detailed  GA  for  Optimization  of  Coefficients  for  Inverse  Transforms 
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3  TEST  RESULTS 


We  conducted  a  series  of  tests  to  demonstrate  the  performance  of  GA-evolved  transforms 
relative  to  that  of  the  Daubechies-4  (Daub4)  wavelet  inverse  transform  [10],  which  may 
be  described  by  the  following  sets  of  floating-point  coefficients: 

h2  =  {0.4830,  0.8365,  0.2241,  -0.1294} 
g2  =  {-0.1294,  -0.2241,  0.8365,  -0.4830} 

Each  test  was  characterized  by  a  particular  combination  of  the  following  parameters: 

1 .  SIGNAL  CLASS.  Each  test  trained  our  GA  using  signals  drawn  from  a  particular 
class.  One-dimensional  signal  classes  used  in  this  study  included  ramp  functions,  sine 
waves,  and  random  signals.  Additional  tests  were  performed  using  2-D  photographic 
images.  These  classes  were  chosen  due  to  their  relevance  to  real-world  applications. 

2.  G  specified  the  (maximum)  number  of  generations  executed  by  our  GA.  Preliminary 
experiments  indicated  that  a  large  G  value  was  necessary  to  allow  our  GA  to  progress 
toward  a  globally  optimized  solution.  For  each  of  the  1-D  signal  tests  performed  for 
this  study,  G  =  10,000.  Since  the  amount  of  data  processed  during  the  2-D  tests  was 
an  order  of  magnitude  greater  than  for  1-D  signals,  it  became  necessary  to  use  only  G 
=  500  generations. 

3.  M  specified  the  population  size  (i.e.,  the  number  of  candidate  solutions  in  each 
generation).  For  each  of  the  tests  performed  for  this  study,  M  =  500. 

4.  Pc  specified  the  percentage  of  individuals  in  the  next  generation  subjected  to  the 
crossover  operator.  Preliminary  experiments  demonstrated  the  benefits  of  a  high 
crossover  rate;  for  this  reason,  each  of  the  tests  performed  for  this  study  used  Pc  = 

100  percent  *  (M-l)/M  =  99.8  percent. 

5.  Pm  specified  the  probability  of  mutation,  as  described  below. 

6.  N  specified  the  size  of  the  training  population  (i.e.,  the  number  of  signals  used  to  train 
our  GA). 

Our  GA  copied  the  best  individual  from  the  current  generation  into  position  0  of  the  next 
generation;  this  individual  was  not  subject  to  subsequent  crossover  or  mutation.  Our  GA 
selected  the  remaining  M- 1  individuals  from  the  current  generation  via  tournaments  of 
three  randomly  selected  individuals.  These  individuals  were  then  probabilistically 
subjected  to  the  crossover  operator  according  to  Pc.  Finally,  each  coefficient  of  these  M-l 
individuals  was  subjected  to  the  mutation  operator  with  probability  PM.  Our  GA 
initialized  Pm  to  2  percent.  If  the  current  generation  failed  to  produce  a  new  globally 
optimal  set  of  coefficients,  our  GA  automatically  increased  Pm  by  1  percent,  up  to  a 
maximum  of  20  percent;  otherwise,  our  GA  reset  Pm  to  2  percent. 

3.1  Class  1:  Ramp  Signals 

One-dimensional  ramp  signals  are  important  for  a  variety  of  applications.  For  example, 
ramp  response  signals  provide  information  about  the  size,  shape,  and  orientation  of 
dielectric  targets  [13].  Tests  1,  2,  and  3  populated  the  training  set  with  N  ramp  signals, 
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each  of  which  contained  50  values.  For  each  signal,  our  GA  initialized  the  starting  value 
a  to  a  randomly  selected  integer  between  255  and  5 1 1 ;  set  the  stopping  value  to  to  a/2; 
and  used  the  following  algorithm  to  determine  the  remaining  values: 


if  (v[i]  ==  to) 

v[i+l]  =  a; 

else 

v{i+l]  =  v[i]  -  1; 

Our  GA  initialized  each  of  the  g2  and  h2  coefficients  for  the  inverse  transforms  in 
generation  0  to  a  randomly  mutated  version  of  the  corresponding  coefficient  from  the 
Daub4  wavelet. 


Table  1  summarizes  the  performance  (as  measured  by  the  total  MSE  for  N  reconstructed 
signals)  of  each  novel  inverse  transform  described  by  a  GA-evolved  best-of-run 
coefficient  set,  relative  to  that  of  the  Daub4  wavelet. 


Table  1.  Ramp  Signal  Test  Results 


Test 

N 

MSE  ( Daub4 ) 

MSE  (evolved) 

%  Improvement 

1 

100 

5505.1 

4838.9 

12.1 

2 

25 

1376.0 

1215.0 

11.7 

3 

10 

552.4 

522.9 

5.35 

These  tests  evolved  the  following  sets  of  coefficients  for  the  optimized  inverse  transform: 


Test  1:  h2  =  {0.3167,  0.8038,  0.3911,  -0.0979} 
g2  =  {-0.1700,  -0.6120,  0.9813,  -0.3859} 
Test  2:  h2  =  {0.3203,  0.81 13,  0.3875,  -0.1054} 
g2  =  {0.1493,  -0.0852,  -0.4700,  0.2132} 
Test  3:  h2  =  {0.4053,  0.81 10,  0.3024,  -0.1052} 
g2  =  {-0.0705,  0.1193,  -0.8903,  0.2180} 


The  results  of  these  tests  demonstrate  the  following  key  points: 


1)  Novel  inverse  transforms  exist  that  outperform  the  Daub4  wavelet  inverse  transform 
for  reconstructing  arbitrary  ramp  signals  that  have  been  subjected  to  lossy 
quantization  operations. 

2)  Our  GA  is  capable  of  automatically  optimizing  coefficient  sets  for  these  novel  inverse 
transforms. 

3)  Training  on  a  larger  population  of  ramp  signals  allows  our  GA  to  evolve  solutions 
that  exhibit  better  generalization  properties. 
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3.2  Class  2:  Sine  Waves 


The  detection  of  periodic  behavior  in  1-D  signals  continues  to  be  a  research  topic  of 
considerable  importance  [14].  Tests  4  through  12  populated  the  training  set  with  various 
types  of  sine  waves.  Each  wave  was  characterized  by  the  following  parameters: 

g  =  gain 
f  =  frequency 
d  =  offset. 


For  these  tests,  each  vector  v  consisted  of  50  sampled  values  defined  as  follows: 

for  (i  =  0;  i  <  50;  i++) 

v[i]  =  g  *  sin  (27if  *  i)  +  d; 

Table  2  summarizes  the  performance  of  GA-evolved  best-of-run  coefficient  sets 
optimized  under  conditions  described  by  various  combinations  of  frequency  (f),  gain  (g), 
and  offset  (d),  relative  to  that  of  the  Daub4  wavelet  described  above. 


Table  2.  Sine  Wave  Test  Results 


Test 

N 

g 

d 

f 

MSE  tDaub4') 

MSE  (evolved) 

%  Improvement 

4 

100 

128 

256 

0...999 

1096.19 

86.69 

92.1 

5 

25 

128 

256 

0...999 

269.00 

20.18 

92.5 

6 

10 

128 

256 

0...999 

124.55 

3.55 

97.2 

7 

100 

128 

0...255 

1000 

5207.62 

5034.13 

3.33 

8 

25 

128 

0...255 

1000 

1203.68 

1182.69 

1.74 

9 

10 

128 

0...255 

1000 

403.67 

397.82 

1.45 

10 

100 

0...127 

256 

1000 

1123.89 

88.19 

92.2 

11 

25 

0...127 

256 

1000 

274.49 

24.90 

90.9 

12 

10 

0...127 

256 

1000 

108.94 

10.30 

90.6 
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These  tests  evolved  the  following  sets  of  coefficients  for  optimized  inverse  transforms: 


Test  4:  h2  =  {0.4875,  0.8279,  0.2109,  -0.1294} 
g2  =  {-0.1280,  -0.2430,  -0.6366,  -0.4881} 

Test  5:  h2  =  {0.4876,  0.8280,  0.2109,  -0.1294} 
g2  =  {0.1 156,  0.2664,  0.9523,  0.3452} 

Test  6:  h2  =  {0.4780,  0.8282,  0.2196,  -0.1305} 
g2  =  {-0.1398,  0.2286,  0.8616,  0.4919} 

Test  7:  h2  =  {0.4447,  0.7768,  0.2569,  -0.0747} 
g2  =  {-0.1050,  -0.3507,  -0.2918,  0.3030} 

Test  8:  h2  =  {0.4448,  0.7849,  0.2592,  -0.0805} 
g2  =  {0.0752,  -0.2607,  -0.7085,  -0.6749} 

Test  9:  h2  =  {0.4927,  0.8024,  0.2175,  -0.0920} 
g2  =  {-0.0649,  0.5076,  -1.0222,  0.2395} 

Test  10:  h2  =  {0.4877,  0.8278,  0.2106,  -0.1294} 
g2  =  {-0.1631,  -0.2723,  -0.9434,  0.5711} 

Test  11:  h2=  {0.4876,0.8280,  0.2108,-0.1294} 

g2  =  {0.1542,  -0.2062,  -0.7879,  0.4552} 

Test  12:  h2  =  {0.4777,  0.8279,  0.2208,  -0.1293} 
g2  =  {0.1337,  -0.2792,  0.7655,  -0.2608} 

Tests  4,  5,  and  6  demonstrated  that  our  GA  was  capable  of  identifying  coefficients  for 
inverse  transforms  that  significantly  outperformed  the  Daub4  inverse  transform  for  the 
task  of  reconstructing  sine  waves  characterized  by  different  f  values.  For  this  signal  class, 
GA-optimized  transforms  were  capable  of  reducing  MSE  in  the  reconstructed  signal  by  a 
factor  of  12  or  more.  It  is  interesting  to  note  that,  while  the  magnitude  of  the  four 
coefficients  comprising  set  h2  were  virtually  identical,  the  four  coefficients  from  set  g2 
exhibited  far  greater  variation,  even  to  the  extent  of  having  opposite  signs. 


Tests  7,  8,  and  9  showed  that,  when  the  training  set  consisted  of  sine  waves  that  differed 
only  according  to  the  offset  d,  little  advantage  was  to  be  gained  from  evolving  novel 
coefficients  for  inverse  transforms.  For  these  three  tests,  the  performance  of  the  inverse 
transforms  described  by  the  evolved  coefficient  sets  improved  upon  that  of  the  Daub4 
inverse  transform  by  an  average  of  only  2.17  percent.  These  tests  also  showed  much 
greater  variation  in  the  magnitude  of  g2  coefficients  than  h2  coefficients,  as  well  as 
differences  in  the  sign  of  those  coefficients. 


Tests  10,  11,  and  12  demonstrated  our  GA’s  ability  to  evolve  coefficients  for  inverse 
transforms  that  significantly  outperformed  the  Daub4  inverse  transform  for  the  task  of 
reconstructing  sine  waves  that  differed  only  in  the  gain  value  g.  For  this  class  of  signal, 
GA-optimized  inverse  transforms  were  capable  of  reducing  MSE  in  the  reconstructed 
signal  by  a  factor  of  1 1  or  more.  As  with  previous  tests,  these  results  showed  greater 
variation  in  the  magnitude  of  g2  coefficients;  indeed,  the  h2  coefficients  evolved  during 
these  three  tests  were  identical  in  sign  and  virtually  identical  in  magnitude,  while  the  g2 
coefficients  varied  in  both  sign  and  magnitude. 
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3.3  Class  3:  Random  Functions 


Tests  13,  14,  and  15  populated  the  training  set  with  one-dimensional  vectors  v  consisting 
of  50  random  values  between  0  and  255  (inclusive).  Table  3  summarizes  test  results 
under  a  variety  of  conditions,  relative  to  the  MSE  produced  by  the  Daub4  inverse 
wavelet. 


Table  3.  Test  Results  for  Random  Signals 


Test 

N 

MSE  ( Daub4 ) 

MSE  (evolved) 

%  Improvement 

13 

100 

12680.4 

12529.2 

1.19 

14 

25 

3271.3 

3230.7 

1.24 

15 

10 

1390.0 

1333.7 

1.04 

These  tests  evolved  the  following  sets  of  optimized  inverse  transform  coefficients: 

Test  13:  h2  =  {0.4781,  0.8248,  0.2285,  -0.1 177} 
g2  =  {-0.1279,  -0.2174,  0.8248,  -0.4728} 

Test  14:  h2  =  {0.4781,  0.8268,  0.2331,  -0.1209} 
g2  =  {-0.1236,  -0.2262,  0.8199,  -0.4763} 

Test  15:  h2  =  {0.4864,  0.8282,  0.2234,  -0.1 192} 
g2  =  {-0.1363,  -0.2135,  0.8263,  -0.4734} 

The  average  performance  (in  terms  of  MSE)  of  the  inverse  transforms  described  by  our 
GA-evolved  coefficient  sets  from  tests  13,  14,  and  15  was  only  1.16  percent  better  than 
that  of  the  Daub4  wavelet  inverse  transform.  This  advantage  remained  nearly  negligible 
as  the  number  of  random  signals  in  the  training  population  increased.  Truly  random 
signals  exhibit  no  particular  pattern;  for  this  reason,  our  GA  was  unable  to  find  and 
exploit  information  common  to  all  of  the  signals  in  the  training  set. 

3.4  Class  4:  Images 

The  goal  of  Tests  16  and  17  was  to  determine  whether  the  GA-based  methodology 
established  by  Tests  1-15  could  be  used  to  evolve  a  transform  that  outperformed  the 
Daub4  wavelet  when  reconstructing  2-D  images  previously  subjected  to  quantization 
error.  Test  16  used  a  training  set  populated  with  N  portrait-like  photographs  similar  to  the 
image  shown  in  Figure  3,  while  Test  17  used  landscape-like  photographs  similar  to  those 
shown  in  Figure  4.  Each  photographic  image  consisted  of  a  5 12-  by  5 12-array  of  red, 
green,  and  blue  (RGB)  color  pixels.  Prior  to  evolution,  our  GA  transformed  each  image 
from  the  training  set  to  the  corresponding  signal  in  the  luminance  (YUV)  domain.  Each 
test  used  a  quantization  step  of  64  (in  effect,  discarding  the  information  contained  in  the 
six  least  significant  bits  of  each  value). 

The  results  of  Tests  16  and  17  are  summarized  in  Table  4.  These  results  clearly 
demonstrate  that  our  GA  was  capable  of  evolving  a  best-of-run  transform  whose 
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performance,  in  terms  of  reduced  total  MSE  over  a  class  of  images,  measurably  improved 
upon  that  of  the  Daub4  wavelet. 


Table  4.  Test  Results  for  2-D  Images 


Test 

N 

MSE  (Daub4t 

MSE  (evolved) 

%  Improvement 

16 

5  (barb,  baboon,  lenna,  susie,  zelda) 

2.30381e+08 

2.18978e+08 

4.95 

17 

5  (airplane,  boat,  fruits,  goldhill,  park) 

1.85457e+08 

1.78302e+08 

3.86 

Figure  3.  A  Typical  Portrait-like  Photographic  Image  (“barb.bmp”)  from  Test  16 
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Figure  4.  A  Typical  Landscape-like  Photographic  Image  (“goldhill.bmp”)  from 

Test  17 


These  tests  evolved  the  following  sets  of  coefficients  for  optimized  inverse  transforms: 

Test  16:  h2  =  {0.4686,  0.8035,  0.2378,  -0.1021} 
g2  =  {-0.2084,  -0.1461,  0.7251,  -0.3935} 

Test  17:  h2  =  {0.4637,  0.8116,  0.2449, -0.1095} 
g2  =  {-0.1680,  -0.1644,  0.7555,  -0.3732} 

Considerable  computational  resources  were  necessary  to  complete  each  2-D  run.  For 
example,  Test  17  required  46  hours,  53  minutes,  59  seconds  of  wall  clock  time  on  a  high- 
end  dedicated  PC.  Access  to  supercomputers  would  allow  future  tests  to  use  much  larger 
populations  (e.g.,  M  =  10,000)  over  many  more  generations  (e.g.,  G  =  5,000),  possibly 
evolving  inverse  transforms  that  exhibit  substantially  better  reconstruction  properties  than 
those  described  above. 


4  CONCLUSIONS 

Collectively,  the  results  of  this  study  suggested  that  the  number  of  coefficient  sets 
capable  of  producing  high-fidelity  signal  reconstructions  under  lossy  conditions  may  be 
much  larger  than  previously  believed.  The  novel  coefficient  sets  evolved  during  this 
study  violated  wavelet  properties  required  for  perfect  reconstruction,  such  as  invertibility 
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and  nonredundancy  [15].  Nevertheless,  the  corresponding  inverse  transforms  consistently 
outperformed  the  Daub4  inverse  transform,  often  producing  significantly  higher  fidelity 
reconstructions  of  periodic  signals  and  images,  as  measured  by  the  percentage  reduction 
in  the  MSE  of  each  reconstructed  signal.  The  results  of  this  study  strongly  encouraged  the 
identification  and  use  of  evolved  inverse  transforms  for  signal  reconstruction  under  lossy 
conditions.  In  particular,  our  results  conclusively  demonstrated  our  GA’s  ability  to 
automatically  identify  novel  sets  of  coefficients  for  inverse  transforms  that  successfully 
reconstruct  various  classes  of  periodic  signals  and  images  under  lossy  conditions  subject 
to  quantization. 

Our  research  will  continue  with  significantly  larger  tests  for  multidimensional  signals  and 
will  include  a  rigorous  mathematic  analysis  of  our  findings. 
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